Skip to content

[fix][broker] Fix the loss of bundle stats data reported to zookeeper, when the updateStats method is executed#19887

Merged
lordcheng10 merged 1 commit intoapache:masterfrom
lordcheng10:fix_PulsarStats
Apr 3, 2023
Merged

[fix][broker] Fix the loss of bundle stats data reported to zookeeper, when the updateStats method is executed#19887
lordcheng10 merged 1 commit intoapache:masterfrom
lordcheng10:fix_PulsarStats

Conversation

@lordcheng10
Copy link
Contributor

@lordcheng10 lordcheng10 commented Mar 22, 2023 •
edited
Loading

Motivation

When the updateStats method is executed periodically, bundleStats will be cleaned up;
At this time, call the getBundleStats method to report the bundle stats data to zookeeper, and the bundle stats data may be lost:

pulsar/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/PulsarStats.java

Lines 105 to 117 in 30d2469

public synchronized void updateStats(
ConcurrentOpenHashMap<String, ConcurrentOpenHashMap<String, ConcurrentOpenHashMap<String, Topic>>>
topicsMap) {
StatsOutputStream topicStatsStream = new StatsOutputStream(tempTopicStatsBuf);
try {
tempMetricsCollection.clear();
bundleStats.clear();
brokerOperabilityMetrics.reset();
brokerStats.reset();
// Json begin

pulsar/pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/ModularLoadManagerImpl.java

Lines 1078 to 1082 in 30d2469

public void writeBrokerDataOnZooKeeper(boolean force) {
lock.lock();
try {
updateLocalBrokerData();

pulsar/pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/ModularLoadManagerImpl.java

Lines 1000 to 1005 in 30d2469

public LocalBrokerData updateLocalBrokerData() {
lock.lock();
try {
final SystemResourceUsage systemResourceUsage = LoadManagerShared.getSystemResourceUsage(brokerHostUsage);
localData.update(systemResourceUsage, getBundleStats());
updateLoadBalancingMetrics(systemResourceUsage);

pulsar/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/PulsarStats.java

Lines 257 to 260 in 30d2469

public Map<String, NamespaceBundleStats> getBundleStats() {
return bundleStats;
}

Modifications

Access to bundleStats should be thread-safe, so both getBundleStats and invalidBundleStats methods should be locked;

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository: lordcheng10#44

@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label Mar 22, 2023
@lordcheng10 lordcheng10 mentioned this pull request Mar 22, 2023
Open
aloyszhang
aloyszhang approved these changes Mar 22, 2023
View reviewed changes
Copy link
Contributor

@aloyszhang aloyszhang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Technoboy- Technoboy- added this to the 3.0.0 milestone Mar 28, 2023
@Technoboy- Technoboy- closed this Mar 28, 2023
@Technoboy- Technoboy- reopened this Mar 28, 2023
@lordcheng10 lordcheng10 merged commit 2dcaf0e into apache:master Apr 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Technoboy- Technoboy- Technoboy- approved these changes

@aloyszhang aloyszhang aloyszhang approved these changes

Assignees

@lordcheng10 lordcheng10

Labels

area/broker doc-not-needed Your PR changes do not impact docs ready-to-test

Projects

None yet

Milestone

3.0.0

Development

Successfully merging this pull request may close these issues.

3 participants

@lordcheng10 @Technoboy- @aloyszhang